Managing Policies

Policy Page

Begin by navigating to the Dynamo Policies tab. This page contains the policy registry - the set of all policies you have created for guardrailing. Through this page, you can edit policies and apply them to models.

Policy Management

You can view more about a policy by clicking the manage policy button. This will take you to the policy instance page, which provides details on the policy definition, training data, and incoming feedback.

Creating a Policy

To create a policy, start by clicking the "New Policy" button. Each policy has different requirements regarding policy creation. To learn more, please see the 'Creating Policies' section.

Training Content Policies

Before applying a content policy, the policy must first be fine-tuned on the synthetically generated training dataset. To train a policy, click the train policy button within the policy management page. The policy will then move to a state of 'Training Queued' and then 'Training Policy'.

Policy Versioning

When training your policy, you will be asked to create a new policy version. Policy versions are used for tracking edits made to policies and enable you to revert changes to an older version.

Applying a Policy

Once a policy is in a "Inactive" or "Ready to Deploy" state, it can be applied to different models. To apply a policy to a model, click apply policy and select the models the policy should be applied to. The policy will then appear in the list of policies for a particular model. To remove a policy from a model, deselect the model the policy should be removed from.

Editing Policies

After a policy has been created, it can be edited by modifying the definition, adding more training data, or leaving feedback on datapoints.

Editing Policy Definitions

Policy definitions can be edited from the Policy Management page. Here, edits can be made to the title, description, or associated behaviors. Allowed and disallowed behaviors can also be added and removed. Once edits to a policy have been made, it is important to regenerate training data to ensure it is aligned to the updated policy.

Manually Adding Training Data

Training data can also directly be added to a policy by clicking the 'Add Data' button in the Training Data tab. Here, it is required to provide the example and compliance status. Once added, the datapoint will be used for model training.

Adding Feedback Data

After deploying a policy to a model, feedback can be added to datapoints in the monitoring log. This feedback can then be applied to the policy and used for policy improvement.

Retraining Policies

After edits are made, it is important to retrain the policy from the Training tab.